Add convenience API key param to remote reindex #135949

PeteGillinElastic · 2025-10-03T16:30:19Z

This adds a source.remote.api_key parameter to the reindex API. This is equivalent to using source.remote.headers with Authorization set to ApiKey <api_key>. It is a convenience for the user.

Note on docs changes: The example request using an API key now uses an applies-switch to show both the pre-9.3 and post-9.3 / serverless versions. There are a few drive-by doc bug-fixes and other improvements to the docs, most notably:

Adding some subsections, as suggested by a tech writer.
Fixing the bug which meant that <OTHER_HOST_URL> was previously not being rendered, because < is a control character.
Repeating the note about using HTTPS for basic auth in the API key section, as we wouldn't recommend sending API keys over plain HTTP either.
A nit, but writing <OTHER_HOST_URL>:9200 is strange, since we have a placeholder URL, and the port is part of the URL, so that's fixed.

elasticsearchmachine · 2025-10-03T16:31:27Z

Hi @PeteGillinElastic, I've created a changelog YAML for you.

github-actions · 2025-10-03T16:33:28Z

🔍 Preview links for changed docs

docs/reference/elasticsearch/rest-apis/reindex-indices.md

github-actions · 2025-10-03T16:33:29Z

ℹ️ Important: Docs version tagging

👋 Thanks for updating the docs! Just a friendly reminder that our docs are now cumulative. This means all 9.x versions are documented on the same page and published off of the main branch, instead of creating separate pages for each minor version.

We use applies_to tags to mark version-specific features and changes.

Expand for a quick overview

When to use applies_to tags:

✅ At the page level to indicate which products/deployments the content applies to (mandatory)
✅ When features change state (e.g. preview, ga) in a specific version
✅ When availability differs across deployments and environments

What NOT to do:

❌ Don't remove or replace information that applies to an older version
❌ Don't add new information that applies to a specific version without an applies_to tag
❌ Don't forget that applies_to tags can be used at the page, section, and inline level

🤔 Need help?

Check out the cumulative docs guidelines
Reach out in the #docs Slack channel

This adds a `source.remote.api_key` parameter to the reindex API. This is equivalent to using `source.remote.headers` with `Authorization` set to `ApiKey <api_key>`. It is a convenience for the user. Note on docs changes: The example request using an API key now uses an `applies-switch` to show both the pre-9.3 and post-9.3 / serverless versions. There are a few drive-by improvements to the docs, including: - Adding some subsections, as suggested by a tech writer. - Fixing the bug which meant that `<OTHER_HOST_URL>` was previously not being rendered, because `<` is a control character. - Repeating the note about using HTTPS for basic auth in the API key section, as we wouldn't recommend sending API keys over plain HTTP either. - A nit, but writing `<OTHER_HOST_URL>:9200` is strange, since we have a placeholder URL, and the port is part of the URL, so that's fixed.

PeteGillinElastic · 2025-10-06T09:53:55Z

Relevant deep-link into the docs preview: https://docs-v3-preview.elastic.dev/elastic/elasticsearch/pull/135949/reference/elasticsearch/rest-apis/reindex-indices#reindex-from-remote

elasticsearchmachine · 2025-10-06T12:03:05Z

Pinging @elastic/es-data-management (Team:Data Management)

Copilot

Pull Request Overview

This PR adds a convenient source.remote.api_key parameter to the reindex API as an alternative to manually setting the Authorization header through source.remote.headers. The change simplifies API key authentication for remote reindexing operations.

Added a convenience api_key parameter that automatically sets the Authorization header to ApiKey <api_key>
Enhanced documentation with better structure, fixed rendering issues, and added version-specific examples
Added comprehensive test coverage for the new API key functionality

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
ReindexRequest.java	Implements the core logic for processing the api_key parameter and validating conflicts with existing Authorization headers
ReindexRequestTests.java	Adds test cases for api_key functionality including conflict detection and header merging scenarios
reindex-indices.md	Updates documentation with improved structure, fixed HTML encoding issues, and version-specific examples using applies-switch
135949.yaml	Documents the enhancement in the changelog

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-06T12:09:42Z

server/src/main/java/org/elasticsearch/index/reindex/ReindexRequest.java

+        if (original.keySet().stream().anyMatch(key -> key.equalsIgnoreCase("Authorization"))) {
+            throw new IllegalArgumentException("Cannot specify both [api_key] and [headers] including [Authorization] key");
+        }
+        Map<String, String> updated = (original instanceof HashMap) ? original : new HashMap<>(original);


The logic mutates the original HashMap which could lead to unexpected side effects. Consider always creating a new HashMap to maintain immutability and avoid modifying the caller's data structure.

Suggested change

Map<String, String> updated = (original instanceof HashMap) ? original : new HashMap<>(original);

Map<String, String> updated = new HashMap<>(original);

I can see why you have done this here to conserve memory when the update isn't required. Apparently CoPilot didn't pick up on that nuance...

Copilot · 2025-10-06T12:09:43Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md

+      "host": "&lt;OTHER_HOST_URL>",
+      "api_key": "&lt;API_KEY_VALUE>"


The HTML entity encoding < appears to be incorrectly used for documentation placeholders. This should likely be <OTHER_HOST_URL> or properly escaped for the documentation format being used.

Suggested change

"host": "<OTHER_HOST_URL>",

"api_key": "<API_KEY_VALUE>"

"host": "<OTHER_HOST_URL>",

"api_key": "<API_KEY_VALUE>"

Not 100% sure what's right here. I can foresee the docs getting hung up thinking this is an HTML tag if using an actual < but also can imagine there being escaping logic that might have this render out as a literal <.

I'm guessing you have tested it out / copied previous examples?

Yeah, I tested this locally. I recommend doing that for docs changes where you're not completely confident, by the way. Running the v3 tool is fairly easy once you've got it installed, and it does some kind of hot reloading so the feedback loop is very fast if you want to experiment.

You can also you can see the preview here: https://docs-v3-preview.elastic.dev/elastic/elasticsearch/pull/135949/reference/elasticsearch/rest-apis/reindex-indices#reindex-from-remote .

Interestingly, it doesn't like it if you replace the > with &rt;, which I was going to do for consistency.

I happened to be discussing this in a thread with the docs team (because I needed advice on the version thing) so I asked there whether there's some other option that's preferred. I'll give them a little while to respond, and if they don't then I'll merge it like this, since it does work.

lukewhiting

LGTM 👍🏻 Nice bit of UX improvement

This reflects the change made in elastic/elasticsearch#135949.

shainaraskas

thanks for the improvements / bug fixes here. added a couple of comments to take or leave.

is this feature releasing immediately for serverless? if it isn't, you might have to separate your docs and code changes so you can add the docs when the feature becomes available.

docs/reference/elasticsearch/rest-apis/reindex-indices.md

shainaraskas · 2025-10-06T17:25:05Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md

+
+### Using an API key [reindex-api-key]
+
+When using {{ecloud}}, it is also possible (and encouraged) to authenticate with the remote cluster through the use of a valid API key:


is this ECH, or does this also work for ECE / ECK?

Suggested change

When using {{ecloud}}, it is also possible (and encouraged) to authenticate with the remote cluster through the use of a valid API key:

When using {{ech}}, you can also authenticate with the remote cluster through the use of a valid API key. This is the recommended method.

It refers to the keys described here: https://www.elastic.co/docs/deploy-manage/api-keys/elasticsearch-api-keys . Actually, I think it also works against serverless with the keys described here: https://www.elastic.co/docs/deploy-manage/api-keys/serverless-project-api-keys although I haven't tested that. (FYI, I didn't write this text, even though the github diff makes it look kind of like I did...)

that makes me think that the keys would work for any deployment type. is there any reason from a code pov that these would be limited by deployment type? stuff at the stack level is mostly extra limited in cloud rather than the opposite. we should make sure, but i'd probably take off the "when using elastic cloud" caveat.

I've tested it with a local dev instance where I created a key via the POST _security/api_key API, and it worked fine. So I will go ahead and remove the caveat, as you suggest.

shainaraskas · 2025-10-06T17:25:29Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md

+::::
+
+
+Be sure to use `https` when using an API key, or it will be sent in plain text. There are a range of settings available to configure the behaviour of the `https` connection.


where are these settings? could downlink to the anchors if they're below. do those settings apply to basic auth?

Yeah, the settings referenced here are the generic SSL settings, they're nothing to do with the auth method used. (This paragraph is a copy-and-paste from the existing text in the basic auth section, because I figured that the advice applied equally to both cases.) I've pushed a commit with the cross-reference.

Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>

PeteGillinElastic

Thanks @shainaraskas . I've taken one suggestion, addressed another comment, there's one comment where I answered your question and await your advice (around the cloud versions).

PeteGillinElastic · 2025-10-06T18:18:04Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md

+
+### Using an API key [reindex-api-key]
+
+When using {{ecloud}}, it is also possible (and encouraged) to authenticate with the remote cluster through the use of a valid API key:


It refers to the keys described here: https://www.elastic.co/docs/deploy-manage/api-keys/elasticsearch-api-keys . Actually, I think it also works against serverless with the keys described here: https://www.elastic.co/docs/deploy-manage/api-keys/serverless-project-api-keys although I haven't tested that. (FYI, I didn't write this text, even though the github diff makes it look kind of like I did...)

PeteGillinElastic · 2025-10-06T18:28:55Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md

+::::
+
+
+Be sure to use `https` when using an API key, or it will be sent in plain text. There are a range of settings available to configure the behaviour of the `https` connection.


Yeah, the settings referenced here are the generic SSL settings, they're nothing to do with the auth method used. (This paragraph is a copy-and-paste from the existing text in the basic auth section, because I figured that the advice applied equally to both cases.) I've pushed a commit with the cross-reference.

…key-sugar

samxbr · 2025-10-06T19:00:11Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md

+
+::::{applies-switch}
+
+:::{applies-item} { "stack": "ga 9.3", "serverless": }


I didn't know about this stack version annotation, I guess it runs the below command against the specified stack and version?

Since we don't have version-specific docs as of 9.x, this makes the page show a selector where the reader can pick the appropriate tab and it shows the relevant snippet. You can see that in the preview.

shainaraskas · 2025-10-06T21:29:52Z

docs/reference/elasticsearch/rest-apis/reindex-indices.md



-Be sure to use `https` when using an API key, or it will be sent in plain text. There are a range of settings available to configure the behaviour of the `https` connection.
+Be sure to use `https` when using an API key, or it will be sent in plain text. There are a [range of settings](#reindex-ssl) available to configure the behaviour of the `https` connection.


…y section, since you can use API keys on pretty much any ES type

…key-sugar

* Add `api_key` param to remote reindex This reflects the change made in elastic/elasticsearch#135949. * Update specification/_global/reindex/types.ts Co-authored-by: Quentin Pradet <quentin.pradet@elastic.co> * Also code-font the `Authorization` header name * make contrib --------- Co-authored-by: Quentin Pradet <quentin.pradet@elastic.co>

PeteGillinElastic added >enhancement :Data Management/Indices APIs APIs to create and manage indices and templates labels Oct 3, 2025

elasticsearchmachine added the v9.3.0 label Oct 3, 2025

PeteGillinElastic force-pushed the reindex-remote-api-key-sugar branch from e26d4db to 5af6d91 Compare October 6, 2025 09:47

PeteGillinElastic marked this pull request as ready for review October 6, 2025 12:02

elasticsearchmachine added the Team:Data Management Meta label for data/management team label Oct 6, 2025

lukewhiting requested a review from Copilot October 6, 2025 12:09

Copilot AI reviewed Oct 6, 2025

View reviewed changes

lukewhiting approved these changes Oct 6, 2025

View reviewed changes

PeteGillinElastic added a commit to elastic/elasticsearch-specification that referenced this pull request Oct 6, 2025

Add api_key param to remote reindex

ca9d0fb

This reflects the change made in elastic/elasticsearch#135949.

PeteGillinElastic mentioned this pull request Oct 6, 2025

Add api_key param to remote reindex elastic/elasticsearch-specification#5425

Merged

shainaraskas approved these changes Oct 6, 2025

View reviewed changes

PeteGillinElastic and others added 2 commits October 6, 2025 19:13

Update docs/reference/elasticsearch/rest-apis/reindex-indices.md

2b67821

Co-authored-by: shainaraskas <58563081+shainaraskas@users.noreply.github.com>

add cross-reference to SSL settings in docs

9fcada6

PeteGillinElastic commented Oct 6, 2025

View reviewed changes

Merge remote-tracking branch 'upstream/main' into reindex-remote-api-…

cca8a15

…key-sugar

samxbr approved these changes Oct 6, 2025

View reviewed changes

shainaraskas reviewed Oct 6, 2025

View reviewed changes

PeteGillinElastic added 2 commits October 7, 2025 09:44

Remove the When using {{escloud}} caveat at the start of the API ke…

0c7fe47

…y section, since you can use API keys on pretty much any ES type

Merge remote-tracking branch 'upstream/main' into reindex-remote-api-…

b89508c

…key-sugar

PeteGillinElastic merged commit d3894ca into elastic:main Oct 7, 2025
34 checks passed

PeteGillinElastic deleted the reindex-remote-api-key-sugar branch October 7, 2025 09:55

	Map<String, String> updated = (original instanceof HashMap) ? original : new HashMap<>(original);
	Map<String, String> updated = new HashMap<>(original);


		### Using an API key [reindex-api-key]

		When using {{ecloud}}, it is also possible (and encouraged) to authenticate with the remote cluster through the use of a valid API key:

	When using {{ecloud}}, it is also possible (and encouraged) to authenticate with the remote cluster through the use of a valid API key:
	When using {{ech}}, you can also authenticate with the remote cluster through the use of a valid API key. This is the recommended method.

		::::


		Be sure to use `https` when using an API key, or it will be sent in plain text. There are a range of settings available to configure the behaviour of the `https` connection.


		::::{applies-switch}

		:::{applies-item} { "stack": "ga 9.3", "serverless": }

Add convenience API key param to remote reindex #135949

Add convenience API key param to remote reindex #135949

Uh oh!

Conversation

PeteGillinElastic commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Oct 3, 2025

Uh oh!

github-actions bot commented Oct 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

github-actions bot commented Oct 3, 2025

ℹ️ Important: Docs version tagging

When to use applies_to tags:

What NOT to do:

🤔 Need help?

Uh oh!

PeteGillinElastic commented Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticsearchmachine commented Oct 6, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 6, 2025

Choose a reason for hiding this comment

Uh oh!

lukewhiting Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lukewhiting left a comment

Choose a reason for hiding this comment

Uh oh!

shainaraskas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shainaraskas Oct 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PeteGillinElastic left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

PeteGillinElastic commented Oct 3, 2025 •

edited

Loading

github-actions bot commented Oct 3, 2025 •

edited

Loading

PeteGillinElastic commented Oct 6, 2025 •

edited

Loading

lukewhiting Oct 6, 2025 •

edited

Loading

shainaraskas Oct 6, 2025 •

edited

Loading